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ABSTRACT 

We introduce a decentralized replication strategy for peer- 
to-peer file exchange based on exhaustive exploration of the 
neighborhood of any node in the network. The replication 
scheme lets the replicas evenly populate the network mesh, 
while regulating the total number of replicas at the same 
time. This is achieved by self adaptation to entering or leav- 
ing of nodes. Exhaustive exploration is achieved by a spiral 
walk algorithm that generates a number of messages linearly 
proportional to the number of visited nodes. It requires a 
dedicated topology (a triangular mesh on a closed surface). 
We introduce protocols for node connection and departure 
that maintain the triangular mesh at low computational and 
bandwidth cost. Search efficiency is increased using a mech- 
anism based on dynamically allocated super peers. We con- 
clude with a discussion on experimental validation results. 

Categories and Subject Descriptors 

C.2 [Computer Commmunication Networks]: Dis- 
tributed Systems 

General Terms 

Design, Algorithm, Experimentation 

Keywords 

peer-to-peer, replication, organization 

1. INTRODUCTION 

In the last few years, the quality (reliability and band- 
width) of internet connections has increased drastically. 
With DSL, users have today an almost permanent high 
bandwidth access to the net. During the download of 
data, the transfert rate is now only limited by the ca- 
pability of the server to manage a large amount of high 
bandwith clients at the same time. Thanks to the peer- 
to-peer (P2P) mechanisms, each user can contribute to 
this data exchange mechanism as a potential server: 



this extends the sharing capability and features some 
fault resilient properties because of the data replication 
and distribution involved. 

Furthermore, P2P architectures allow users to share 
resources. Few existing architectures allow CPU cy- 
cle sharing [5], but most of them are designed to share 
memory and to behave as distributed databases. Data 
replication keeps information available when a node fails 
or leaves the network. Replication is useful for scalabil- 
ity because if a lot of users want to have simultaneous 
access to the same information, a single peer may not 
be able to supply all the queries. 

Several architectures have been proposed for P2P net- 
works. Napster [15], for instance, is a centralized P2P 
network that was very popular in the early 00's. How- 
ever the use of a central repository to answer queries 
makes this system poorly scalable and vulnerable to 
failure. Many others P2P systems do not rely on a cen- 
tral server: they are decentralized, thus they are very 
scalable, and fault tolerant. 

Structured P2P networks associate network topology 
and location of data. Most of them implement a Dis- 
tributed Hashtable (DHT) [20, 16, 17, 23, 14] and pro- 
vide one basic operation: mapping a key to a specific 
node. This is performed by using a distributed hash 
function. Content routing is used to forward the key 
to the corresponding node. This strategy is very well 
suited for rare information retrieval (i.e. with a low 
number of replicas). Most of the structured P2P net- 
works can retrieve a key in 0(log(N)) messages in a 
network containing N nodes. Their main limitation re- 
lies in poor performances for ranged and approximative 
queries, because hashing destroys the order on keys. 
Oceanstore [10], Cooperative File System (CFS) [4] or 
Glacier [6] are examples of storage systems, built on 
top of DHTs and on replicated data to provide high 
availability. 

Conversely, unstructured P2P networks do not fea- 



ture any constraint between the location of data and 
the network topology. Gnutella [3], eDonkey 2000 [8], 
KaZaA [12] and Freenet [2] are examples of such work- 
ing architectures. The previous versions of Gnutella use 
flooding: this generates a number of messages that in- 
creases exponentially with the radius of the search. The 
trafic generated by this mechanism limits the scalabil- 
ity. To solve this problem, random walk, and variants 
such as k-random walk [13], have been proposed. If a 
node receives a query and cannot answer it, the query 
is forwarded to a randomly chosen neighbor. When a 
query is answered, it is then back-propagated to the 
node that initiated the query using the inverse path. 
Because random walk does not rely on the content of 
the query, much more complex queries can be processed, 
such as approximative or range queries. 

A Gnutella-like system has been proposed in [1]. The 
system relies on measures performed by [18] that show 
the hctcrogcnity in capabilities of the peers involved in 
an unstructured P2P network such as Gnutella. This 
system uses a biased random walk that visits the nodes 
with a higher degree first. Each node has pointers on 
data hosted by its neighbors, and each new node that 
enters the network tries to connect to a high-degree 
node. These features increase the overall performance 
of the system by three to five orders of magnitude. 

In Freenet, routing efficiency increases as the network 
becomes more and more organized. However, it uses 
content routing of keys, which means that a user looking 
for particular data needs the knowledge of the key of the 
data. As for structured P2P systems, it is very costly 
to perform range or approximative queries. 

In popular P2P systems like Gnutella, eDonkey 2000 
or KaZaA, replication is performed by users that down- 
load copies of files on their own computer. This owner 
replication scheme is well suited for popular data, how- 
ever rare data remains very hard to reach. 

This paper presents the design of a P2P file shar- 
ing system called Adapnet that offers high availability 
in exchange for storage space offered by each user. Be- 
cause it does not rely on hashing like DHT's or Freenet, 
our system can handle very complex queries including 
ranged or approximative queries. A loose structure al- 
lows adaptive replicating algorithms to execute while a 
super-peer architecture [22] speeds up query answering 
and makes the system scalable. 

Section 2 briefly describes our system. Section 3 
presents the adaptive replication algorithm built on a 
dedicated topology and a well-suited marching algo- 
rithm called spiral walk. Section 4 describes the ar- 
chitecture used for fast query answering. In the last 
section, we evaluate Adapnet on a simulated network 
designed on real P2P statistical features [18, 7]. 

2. OVERVIEW 



In this article, we present a data P2P architecture 
designed to offer high availability. In this framework, 
users offer some disk space used by the system to store 
information replicas. The proposed system is designed 
to manage rare data: therefore mostly replicas of rare 
data are created and distributed across the network. 
Replicating rare data keeps a fixed amount of informa- 
tion reachable even if a significant amount of nodes are 
offline. 

We introduce an algorithm designed to keep the dis- 
tribution of the replicas as evenly spaced as possible to 
ensure that they remain in the smallest neighborhood 
possible of any node. The replication scheme involves 
an autonomous mechanism that adapts the number of 
replicas and their distribution in the network to the 
number of active nodes. For each piece of data, the 
quantity of replicas is lincary proportional to the over- 
all number of nodes in the network. This quantity in 
the system is globaly unknown. 

The replication scheme we propose relies on the ex- 
haustive exploration of the neighborhood of any node. 
While some structured P2P architectures allow an easy 
exploration of this neighborhood, they only support 
queries on key: range or approximative queries are very 
costly or not supported at all. These topologies are del- 
icate to maintain when nodes leave the network without 
notification. 

Unstructured P2P architectures allow wide query ex- 
pressivity but are slower that structured P2P networks 
to locate information. Using the heterogeneity among 
peer capabilities in the network, super peer networks 
can be built to speed up information retrieval. Because 
there is no heavy topological properties to maintain, 
these networks are very well suited for highly dynamical 
environments. However, this lack of topological prop- 
erties makes the exhaustive exploration only possible 
with flooding, therefore very costly. 

We present in this article a marching algorithm called 
spiral walk that explores the neighborhood of any node 
generating a number of messages lineary proportional 
to the number of visited nodes. This algorithm needs 
a dedicated topology to perform. More precisely the 
topology required is a triangular mesh on a closed sur- 
face. 

We describe simple protocols for node connection or 
topology repairment after a node failure or a node de- 
parture that maintain this loosely structured topology. 
Nodes connection or failure are handled by the network 
with constant processing time operations according to 
the overall number of nodes. The average additionnal 
connectivity introduced is low : nodes have an average 
degree of six. 

Our system uses the heterogeneity featured in un- 
structured P2P networks described in [18] to speed up 
information retrieval. Peers with higher ressource (band- 
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width, CPU, memory, ...) additionnaly compose a super- 
peer network used to efficiently forward queries. Its 
random topology makes this super-peer network have a 
low diameter. Super-peers hold indexing information of 
other nodes, allowing fast query answering. 

3. ADAPTIVE REPLICATION 
3.1 Replication scheme 

3.1.1 Overview 

We introduce a replication scheme that adapts the 
number of data replica without knowing the overall 
number of active nodes in the network. 

• Let 5(n\,n2) be the distance between two nodes 
m and n 2 in the network. S(ni, n 2 ) is the number 
of hops of the shortest path that links n\ to n 2 . 

• Let R ntr = {ni\5(rii, n) = r}. 

• Let di be a data and D n be the set of data hosted 
on a node n. 

Efficient storage management requires the optimiza- 
tion of the number of replicas. We propose a self or- 
ganizing framework in which the replicas move across 
the network, repulsing each other until a near to op- 
timal placement has been found to fulfill the bounded 
distance requirement. 




Figure 1: di replicas repulse each other to pop- 
ulate evenly the network. 

This scheme involves competition between replicas 
and a cloning mechanism, which lets replicas homoge- 
neously populate any region of the network. The main 
idea is very similar to the algorithms used in the arti- 
ficial life or automata processing [11]. The set of repli- 
cas can been seen as a population sharing a vital space 
by means of reproductive rewards and death penalties. 
This strategy is decentralized allowing the replicas to 
act as autonomous agents. This population reacts at 
runtime and adapts its distribution accross the network 
to the changes of environnement such as topological 
variations (entering or leaving of nodes) or resources 



variations (change of local storage space). Since the 
distribution involves a metric to compute a distance, 
we propose a dedicated topology, as well as a spiral 
walk strategy designed to fully and efficiently scan all 
the nodes below a maximal distance of any given node 
n. 

3.1.2 Replication scheme 

We present a replication scheme that is fully dis- 
tributed and that maintains a number of replicas pro- 
portional to the size of the network, without knowing its 
size. The system mostly creates replicas of rare data. 
Replicas are spread accross the network, and periodi- 
cally, a score related to the local density of replicas for 
a given di is computed. According to this score, the 
replica are destroyed, moved, duplicated or kept on the 
node, as described in algorithm 1. Because the consid- 
ered topology is closed, replicas can repulse each other 
and become evenly spread accross the network. 

Algorithm 1 Replication from node n 
for all d e D n do 

S <- Sc{n,d,r,<D) 

if S > maxScore then 
Remove d from n. 

else if S = then 

Let R n _ r ,t a random subset of R n>r of size t. 
Replicate d on the node n p <G R n ,r,t which mini- 
mizes Sc(n p , d, r, 0). 

else 

Let nminl^nt G R n ,i, 
Sc(n min ,d,r,{n}) < Sc(rii,d,r,{n}). 
Move d to n min . 
end if 
end for 



• I(rii, d) = 1 if d € D n ., otherwise 

• Sc{n, d, r, SI) = J2 niGx H^d).(r - 5{n, m) + l) 2 

• ft is the set of nodes to ignore for score computa- 
tion 

• X = {ni\S(n, n l ) < r} - SI 

The choice of maxScore is a trade-off between reac- 
tivity to the changes in the network and stability. A 
low value for maxScore makes the number of replicas 
oscillate when a small variation of the number of nodes 
occurs. A higher value for maxScore reduces the oscil- 
lations but lowers the reactivity of the system to adapt 
to the variation of its environment. 

t is an exploring constant. The higher t, the bet- 
ter the exploration for the placement of new replicas, 
however the associated computing cost also increases. 

r is the repulsive radius. The higher r, the lower the 
density of replicas. 
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Carrying the whole data during the scan of the neigh- 
borhood of a node can be costly. Therefore only hash 
values of those data are carried and compared. More- 
over, many data replicas can be scanned simultaneously 
Scanning only a part of the neighborhood leads to an 
approximative score value and produces oscillation of 
the quantity of replicas, even in a stable environment. 
This increases the convergence time of the replication 
algorithm. To overcome this limitation, we propose an 
efficient and exhaustive exploration of the neighborhood 
using spiral walk, described in section 3.3. 

When a replica has to be moved or replicated, the 
algorithm 1 makes the assumption that there will be 
enough space on the node to perform the replication. 
However, this cannot be garanteed in practice and there 
is a need to delete replicas when the cache of a node is 
full. This is performed by computing the score for all 
replicas and deleting the ones having the highest scores. 
If the score is the same for all replica (for cxcmplc null) , 
then replicas choosen at random are deleted from the 
cache. 

3.2 Support topology 

Replicas repulse each other on a surface. A ring 
topology could be maintained at low cost, however repli- 
cas would diffuse lincary. On the other hand, perform- 
ing this repulsion on a higher dimensional space would 
speed up this diffusion, but it would be more costly to 
maintain the topology. The network topology we por- 
pose is a triangular mesh on a closed 2-manifold offering 
a good trade-off between speed diffusion and mainten- 
ing cost. 

Using the Gauss-Bonnet theorem [19] and the Eulcr 
characteristic, we deduce that the local curvature of the 
network topology around a node n is: 

• positive if n has five neighbors or less. 

• null if n has exactly six neighbors. 

• negative if n has seven neighbors or more. 

As more and more nodes connect to the network, the 
average node degree tends to six, no matter the value of 
the initial average degree. This reduces the overall cur- 
vature of the topology. However, connection and failure 
recovery protocols locally deform the topology. Unfor- 
tunately, the presence of deformations in the topology 
alters our replication algorithm, as it reduces the pos- 
sibilities for organizing replicas, and leads to a less ef- 
ficient placement of replicas compared to a nearly flat 
topology. Replica placement is a key factor for the per- 
formances of the system, therefore we propose an algo- 
rithm to flatten the topology. 

We assume that each node on the network has a 
unique identifier, this could be for instance the ip ad- 
dress. We assume there is an order relationship between 



all identifiers. Nodes refresh their knowledge of their 
neighborhood by sending the identifier of their neigh- 
bors to their other neighbors. This can be viewed as a 
more elaborated ping message. This knowledge stored 
on nodes speeds up common neighbors discovery for a 
pair of neighbor nodes. 

3.2.1 Topological invariant 

In the triangular mesh, each edge belongs to exactly 
two triangles. More precisely : 

8{n 1 ,n 2 ) = 1 => l-Rni.i n R n2 ,i\ = 2 

In other words, if n\ and n 2 are neighbors, then they 
have exactly two neighbors in common. This leads to a 
network in which any node has a minimal degree of four. 
The only network that fits those requirements with the 
lowest minimal degree is a tetrahedron. 

3.2.2 Node connection 

A node n joining the network contacts a random node 
n\ in the network. The way this random node n\ is 
obtained can be achieved in various ways (for instance 
with a central server registering few nodes taken at ran- 
dom) and is not addressed here. A node n 2 , neighbor 
of m is then taken at random. Connection between 
m and n 2 is terminated and new connections are cre- 
ated between n, n\ and n, n 2 . Finally, n contacts the 
two neighbors common to n\ and n 2 , according to the 
property introduced in section 3.2.1, and creates a con- 
nection with them, as described in figure 2. 




Figure 2: Connexion of a new node to the net- 
work. 

This connection protocol adds one node and three 
connections to the network, bringing the average node 
degree in the network closer to six. This increases local 
curvatures in the topology, as n has a degree of four and 
ri3 and n± have their degree increased by one. That is 
why the node n then tries to acquire new neighbors 
by flattening the topology locally to its neighborhood. 
This flattening process is described in section 3.2.4. 

3.2.3 Node disconnection 

We make the assumption that a node leaving the net- 
work is equivalent to a failure of this node. A failure 
on a node n is detected by its neighbors when they do 
not receive a ping message of n within a given time- 
out. If a failure is detected, then the topology needs 
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to be repaired in order to maintain a triangular mesh. 
When the failure of a node is detected by its neighbors, 
the node having the lowest identifier launches a repair 
agent. This is possible because each neighbor of the 
failing node is aware of all other neighbors. This agent 
creates connexions between neighbors of the node that 
has left, repairing the hole in the topology as shown in 
figure 3. 




Figure 3: Topology repair after node failure. 

There are two kinds of holes in the mesh: some are 
repairable whereas others are not. If the missing node 
has at least one neighboor with a degree of four, then 
the hole in the mesh cannot be fixed because the pre- 
viously described procedure may lead to triangles with 
nodes having a degree of three, breaking the invariant 
property described in section 3.2.1. In this case, a ran- 
dom walker is sent through the network to find a node 
that can replace the failing node. The replacing node 
must leave a repairable hole in the mesh, which means 
that all of its neighbors must have a minimal degree of 
five. 

Even if all neighbors of the failing node still have a 
degree of four, the repair agent may break the topo- 
logical invariant. If the repair agent does not have a 
solution to repair the hole, it breaks all the connections 
previously created and another node in the network is 
moved to fill the gap. 

3.2.4 Topology flattening 

Local deformation of the topology reduces the injec- 
tivity radius of the spiral walk. Node connections or 
node failures can lead to a locally curved topology. A 
flat topology allows replicas to self-organize in a near- 
optimal solution according to their dispersion strategy. 
Therefore the topology needs to be dynamically and 
locally flattened. This flattening process is performed 
each time a node joins the network or after a topology 
repairemcnt . 

For a given node, if it has a degree greater than six, 
then the neighbor with the highest degree is selected. If 
the sum of their degree is greater than the sum of the 
degree of their two common neighbors, then the two 
nodes break their connection and their two common 
neighbors connect together, as shown in figure 4. This 
is performed only if the creation of the link between the 



two common neighbors does not break the topological 
invariant: the two common neighbors must have only 
two common neighbors. 




Figure 4: Topology flattening. 



When a node joins the network, it has a degree of 
four. This flattening process increases the degree of new 
nodes, and reduces the overall number of nodes having a 
degree of four. Flattening the network topology makes 
also the network repairment easier and simplifies the 
scanning of node at defined distance as described in the 
next section. 

3.3 Spiral Walk 

A biased random walk has been introduced by [9]. 
This walk is designed for sensor networks which are a 
particular case of unstructured P2P networks. The bias 
introduced in this random walk favors the visit of nodes 
near the source before visiting those farther away. How- 
ever, this kind of walk cannot guarantee that all nodes 
within a given radius will be visited. 

In a sensor network, peers can connect to each other 
if they are close enough. In an unstructured P2P net- 
work built over internet topology, peers can connect to 
any other peer wherever it is, which makes it possible 
to control the virtual network topology. Using the algo- 
rithm proposed by [9] on a regular triangular mesh (i.e. 
all nodes have a degree of six), one can notice that this 
algorithm performs a compact exploration. This kind 
of walk can perform very well when nodes have common 
neighbors. 

We detail a spiral walk (SW) algorithm that performs 
a compact and exhaustive exploration of the neighbor- 
hood of a node on a triangular mesh, fullfilling the prop- 
erty described in section 3.2.1. Given a graph G and a 
source node n € G, SW visits all nodes in the neigh- 
borhood of n within a given radius. This radius refers 
to the minimum number of hops of the shortest path 
between n and other nodes. 

The number of generated messages is linearly propor- 
tional to the number of visited nodes; unlike strategies 
such as flooding that generates a number of messages 
exponentially increasing with the radius of the explo- 
ration. 

3.3.1 Spiral walk principle 
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Spiral walk visits all nodes within a given distance 
of the node that initiates the walk. Spiral walk can be 
viewed both as a local space filling curve and as a kind 
of k-random walk. It is a breadth-first search (BFS) 
exploration: this property ensures that the exploration 
within a given radius is performed exhaustively in a 
bounded number of hops. 

This algorithm is designed for a triangular mesh on 
a closed 2-manifold with the following property: two 
neighboring nodes in the network have exactly two com- 
mon neighbors, as described in section 3.2.1. This prop- 
erty allows the entire neighborhood of a node to be se- 
quentially visited. 




----- Rns.l 

Rns,2 

Rns,3 



Figure 5: Three nested rings of a spiral walk. 
Pi is the sequence of nodes in R ns a used and 
is the sequence of visited nodes while exploring 

We assume that the network topology is stable and 
that no node leaves or enters the network during the 
walk. 

The main idea is to build rings starting from the node 
n s launching the walk. Each nested ring is built in ref- 
erence to the previous one. Using the notations defined 
in section 3.1.1, R ne ,i is the i th ring of the spiral walk 
starting from n s . R ns ,o is the first ring containing only 
the node n s initiating the walk. R ns ,max is the last ring 
of the walk, max being the radius of the walk. 

Having the following property: Vn £ R ns s-i, \R n ,i H 
Rn 3 ,i-i\ = 2, nodes in R na ,i-i can be sequentially vis- 
ited, since two neighbors £ R ns ,i-i have one common 
neighbor in R ns ,i~2 and one common neighbor in Rn B ,i- 
The way i?„ si j + i is built, knowing R Us .i-i and visiting 
Rn s ,i is described in algorithm 2. 

Each time a node £ R ns .i is visited, all its neighbors 
that are not in R ns ,i-i or R ne ,i are recorded in R n ^ i+ i. 
When R ns .i has been exhaustively visited, all R ns .i+i 
are known. Spiral walk ends when the walker reaches a 
given TTL (Time To Live) given in time or hops units, 
or when the walker reaches a given radius. 



Algorithm 2 Building next ring. 

p^n : n £ R ns s-i, \R n ,i H Rn s ,i\ > 1 
q^n : n £ R ne ,i l~l Rp,i, \Rn,i l~l Rn e ,i-i\ > 1 

P <— R net i-1 
Rn 3 ,i+1 <— 

while P ^ do 

Rn s ,i+i <— ^ 8 ,i+iU{ti : n £ R qt i\Rn s j--\_L)R nst i} 
q <— n : n £ Rn s ,i U Rq,i U R P ,i 
if n J?„ si j_i| > 1 then 
repeat 

P^P\{p} 

p^n : n £ R Us ,i-i n R p ,i n R q .i 
until \Rp tl n Rn s ,i\ > 1 
end if 
end while 



3.3.2 Walker's return 

After having completed the exploration, the spiral 
walker returns to the initial node by a new way. During 
its exploration, it records in a hashtable the hop dis- 
tance to the source of each node it visits. Using this 
information it can go back to the source using this gra- 
dient of distance. The length of the return path is equal 
to the radius of the walk. 

Another interesting feature is that this property can 
be used to send back walker each time interesting in- 
formation is found, so that source node can receive an- 
swers before the spiral walker finishes its exploration. 
As back walkers do not necessarily take the same path, 
this can give information gradualy, spreading network 
traffic over the whole neighborhood of the source node. 

This generates a low number of messages everytime 
information is back propagated. Let k be the number 
of answers and N the length of the walk to find those 
k answers. Because of the short cut, the number of 
message using spiral walker will be 0{k^/~N). On the 
other hand, using random walk, the number of messages 
will be O(k.N). 

3.3.3 Locally curved space 

The rules defined for node connection and the process 
involved in the topology rcpaircmcnt arc very simple 
and the cost to maintain this topology is very low. How- 
ever, these updating rules can lead to a curved space 
that reduces the injectivity radius of the walk: some 
nodes may be both distant in the ring and neighboors 
in the topology. This creates shortcuts for the building 
of the next ring and may lead to unexplored area, we 
call these phenomenons eyes, as shown in the figure 6. 

Although we present a way to reduce the curvature of 
the topology, spiral walk may still visit the same node 
twice, especially if the radius of the walk is large. Be- 
cause all nodes in R Us ,i are visited sequentially and all 
nodes in R nsti _i are known when i?„ si j is visited, we can 
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detect if a node in Rn B ,i has more than two neighbors 
in R ns .i. If it occurs, then this means that the space is 
locally curved. This problem is fixed by spawning spiral 
walkers accross these eyes. 




Figure 6: Eyes appearing during the walk. 

Each spawned spiral walker inherits the memory of 
the original one and can go back to the source node 
that has initiated the walk using the gradient of dis- 
tance as described in section 3.3.2. The only difference 
between the two spiral walkers is that the former will 
build larger and larger rings, whereas the latter will 
build smaller and smaller ones. This allows us to deal 
with extreme configurations of the topology where some 
eyes may recursively contain other eyes. 

When these configurations occur, our algorithm can 
be viewed as a kind of k-walker, with k — 1 eyes. The 
advantage of this approach is that, thanks to the proper- 
ties of the network topology and the gathering of infor- 
mation about visited nodes, all spawned walkers will not 
visit the same nodes, whereas in traditional k-random 
walk this may happen. 

It creates a parallel search, each spawned walkers hav- 
ing its own domain to explore, but it alters replicas 
placement and leads to a worse organization than on a 
flat topology. 

3.3.4 Dealing with very dynamic environment 

Previously, we made the assumption that the network 
is stable during a spiral walk. Here, we show how change 
of the network topology during the walk can be dealt 
with. The solution provides an approximative walk and 
the exhaustivity of the walk is lost. 

The first solution uses the previously described spi- 
ral walk algorithm. Each time an interesting piece of 
information is found, a walker is sent back. When the 
walker detects an anomaly in the building of its rings, 
it stops. This solution provides a partial answer if a 
node in Ri-i, Ri or disconnects while the walker 
is in Ri. Otherwise, the algorithm still works as it uses 
a gradient mccanism very resilient to node departure 
to go back to the source node that initiated the spiral 
walk. 

The second solution uses the algorithm proposed by 
[9]. This walk can be performed in very dynamic en- 
vironments. However, because of its random behavior 



in non dedicated network topologies, it does not ensure 
the exhaustivity of the walk and reduces the efficiency 
of the replication algorithm. 

4. SEARCH PROTOCOL 

4.1 Introduction 

The cost of random walk and variants for the routing 
of queries is generally of 0(log(N)) for a network with 
TV nodes. This seems to be very high when compared 
to most standard routing P2P mechanisms, but the dif- 
ference is mainly related to the heuristics used. Since 
no assumption is made in our case for the kind of query 
performed, the routing cost is balanced by the fact that 
ranged or complex queries can be performed along the 
walkers path. 

Using the super-peer paradigm, as suggested by [22], 
can speed up the search by a factor proportional to 
the average number of sub-peers per super-peers. For 
instance, if one percent of nodes in the network have 
100 times more resources than the average of nodes, the 
search space can be reduced by a factor of 100. In the 
following sections, we describe the layout of our system 
and introduce the rules for the creation of super-peers. 

4.2 Search layout 

A super-peer is a node in the network that has above- 
average resources. A super-peer is connected to other 
super-peers, allowing to efficiently forward queries in 
the whole network, assuming the super-peer network is 
strongly connected. Moreover, a super-peer manages 
many sub-peers, the number of managed sub-peers be- 
ing proportional to the super-peer resources. In most 
existing unstructured P2P networks, sub-peers are called 
clients because they are only connected to super-peers. 
We use the term sub-peers because all nodes in the net- 
works are not connected to only super-peers, but also 
to sub-peers. 























\ .1 





Figure 7: System design : the triangular mesh 
allows compact exploration at low cost while 
high connectivity nodes are used to increase 
search efficiency. 

Each super-peer is responsible for at least one other 
peer (including itself) and each peer has one super-peer, 
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as shown in figure 7. 

4.3 Query routing 

In order to speed up query routing, each node indexes 
the data locally stored and sends a copy of the indexed 
information to its related super-peer. 

A query initiated by a peer is forwarded to its super- 
peer, with a Time To Live (TTL) strictly positive. The 
super-peer first handles the query for each of its other 
sub-peers. Then TTL is reduced by one and the query 
is forwarded to a random neighbor in the super-peer 
layer. Each time a super-peer receives a query, the 
query is handled for its sub-peers and forwarded to an- 
other random neighbor in the super-peer layer until its 
TTL reaches 0. 

4.4 Super peer management rules 

Super-peers are the entry points for nodes wishing 
to join the network. It can be difficult for a node to 
contact a super-peer of the network, that is why super- 
peers never refuse new clients. To avoid being over- 
loaded, super-peers regulate their number of sub-peers 
themselves. 

4.4.1 Super-peer creation 

If a super-peer m has to manage too many sub-peers, 
one of its sub-peers n 2 is promoted super-peer to re- 
balance a too heavy load. The sub-peer selected is 
the one having the highest capabilities among all sub- 
peers of m. The new super-peer n 2 contacts its nearest 
neighbors in the triangular mesh and becomes their new 
super-peer. The number of nodes contacted is propor- 
tional to the resources of n 2 . Nodes are contacted by 
launching a spiral walker in the triangular mesh. 

This scheme ensures that all sub-peers of n 2 are close 
in the triangular mesh. Because replicas of a given piece 
of data repulse each other in the triangular mesh, a 
super-peer will unlikely have twice the same piece of 
data from different sub-peers. 

In order to efficiently speed up information retrieval 
and to keep the super-peer network resilient to frag- 
mentation, the new super-peer n 2 has to aquire new 
neighbors in the super-peer network. This is achieved 
by sending a random walker in the super-peer network 
through n\. Super-nodes visited by the random walker 
contact n 2 and become its new neighbor in the super- 
peer network. Only super-peers having a low degree in 
the super-peer network try to connect to n 2 , as the fail- 
ure of high-degree super-peers may lead to a fragmented 
network. 

4.4.2 Super-peer failure 

We make the assumption that a super-peer never no- 
tify its departure from the network. When a super- 
peer loses a neighbor in the super-peer network and 



its degree becomes low, it can acquire new super-peer 
neighbors, using the strategy described previously for 
super-peer creation. 

However, the absence of notification at the sub-peer 
level raises problems for the sub-peers of the failing 
super-peer because they cannot send their queries ac- 
cross the super-peer network. Since they do not have 
a super-peer, these nodes launch their queries accross 
the triangular mesh. Those queries are forwarded ran- 
domly in this part of the network until a node having a 
super-peer is encountered. The query is then forwarded 
at random in the super-peer network. 



Algorithm 3 Regulation from super node sni 
while true do 

sn 2 <— sni.getRandomSuper Neighbor () 

sni.numSubPeers()+sn2.numSubPeers() ^ 
sni.capabilitiesQ 

for all n e sn 2 .getSubPeers() do 

sni .addSubP eer{n) 

n.setSuperNode(sni) 
end for 

Downgrade sn 2 as normal peer. 

if sni.hasFew Super Neighbor s{) then 

Find new super neighbors for sni. 
end if 
end if 
Wait x sec. 
end while 



Because the sub layer is strongly connected, some 
sub-peers of the failing super-peer have neighbors in 
the topology with a different super-peer, as described 
in 4.4.3. We call such sub-peers borderline-peers. When 
they lose their super-peer, all borderline-peers create a 
link to the super-peer of one of their neighbors. All their 
neighbors which have lost their super peer and are not 
borderline peers become borderline-peers, as they now 
have a neighbor in the triangular mesh with a super- 
peer. 

4.4.3 Load balancing 

When there are more departures than arrivals in the 
network, the overall number of node decreases. Super- 
peers have on average a higher uptime than other peers. 
To keep the number of super-peers proportional to the 
number of peers on average, there is a need to down- 
grade super-peers. This is achieved by the algorithm 
3: neighboring super-peers periodically send a message 
containing their load measure to each other. If one of 
them can handle the whole load, it recovers all sub- 
peers of its neighbor and the other is downgraded as 
normal-peer. 

As shown in figure 7, all nodes have a set of neigh- 
bors in the sub layer of the network. Two neighbors in 
the sub layer graph can have a different super-peer. In 
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this configuration, the two neighbors periodically com- 
pare the load measure of their super-peer. They can 
become the sub-peer of the super-peer of their neighbor 
to balance the load between super-peers, as described 
in algorithm 4. 



Algorithm 4 Bottom-up regulation from node n 
while true do 

sn\ *— n.getSuperNodeQ 

sn 2 <— n.getRandomNeighbor().getSuperNode() 

7 t sn-\.numSubPeers() — l 

loadi < i r^rr- — H — 

L sn-i .capabiLities{) 

^ ^ sri2.numSubPeers()-\-l 

2 sn2-capabilities() 

if loadi > load 2 then 

sni .removeSubPeer(n) 

n.setSuperN ode(sri2) 

sri2.addSubPeer(n) 
end if 
Wait x sec. 
end while 



5. EXPERIMENTS 

Experiments have been performed on a dedicated sim- 
ulator we have developped in Java. At initialization, 
the network is made of 12 nodes and the topology is an 
icosahedron, with all nodes having a degree of five. The 
average node degree gets closer to six as more nodes are 
added to the network. 

In order to simulate heterogenity in peers, nodes are 
given capabilities following the distribution in figure 8. 
These capabilities reflect the number of connexions or 
load of work a node can handle. We believe this dis- 
tribution is close to real distribution, as shown by the 
measurements in [18, 7]. More precisely, this capabil- 
ity is used to compute the amount of data a node can 
host, or the number of sub-peers it can manage when 
promoted super-peer. 




oJ — — I — 

1 1 1 00 1 000 

Capability level 



Figure 8: Simulated capability repartition 
among peers in the network. 



5.1 Replication 

To evaluate the influence of the regularity of the Adap- 
net mesh, we compare the results with the same scheme 
performed on an optimal mesh. By recursively sub- 
dividing an icosaedron, we can generate a triangular 
geode. This can be viewed as an optimal closed trian- 
gular mesh mapped on a sphere. In this section, we re- 
fer as the experimental topology the network obtained 
using the protocols described in section 3.2 for node 
connection or departure. The optimal topology, start- 
ing with an icosaedron, is a triangular geode. In order 
to perform a comparison, the geode we have simulated 
contains 40962 nodes, so does the simulated experimen- 
tal topology. 
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Figure 9: Evolution of the quantity of replicas 
for one piece of data, r = 25, maxScore — 250 and 
t = 16. 

The graph in figure 9 shows the evolution of the num- 
ber of replicas for one piece of data on the experimental 
topology and on the geode. Both topologies are static, 
no node joins or leaves the network. We assume in this 
experiment that there is always some available space in 
the cache of nodes and that caches are never purged. 

Initially, a piece of data is put on a random node. 
Then during each iteration, each node in the network 
having a replica of the data scans its neighborhood and 
decides if it must create another replica, move the one 
it is hosting on another node, delete it, or do nothing , 
as described in algorithm 1. 

We can see on figure 9 that, given a stable topol- 
ogy, the replication algorithm converges. The number of 
replicas created or moved drops to zero. More replicas 
are created on the geode: this is because the geode has 
a higher diameter than the topology in practice. Irreg- 
ularities on the topology make the algorithm converge 
faster, we believe it is because it reduces the number of 
sub-optimal configurations. 

Figure 10 shows the impact of the repulsive radius on 
the global amount of replicas. It confirms that a perfect 
mesh has a higher diameter than an irregular one, given 
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and observe the evolution of the number of replicas for 
one piece of data. 



15 20 

repulsive radius (r) 



Figure 10: Quantity of replicas according to 
the repulsive radius r for one piece of data. 

maxScore = 10 x r and t = 4. 



the same amount of nodes. As a consequence, less repli- 
cas are requiered to populate the whole network. As the 
repulsive radius r increases, the geode topology propor- 
tionally includes more replicas than the experimental 
topology. 

Irregularities into the experimental topology lowers 
the amount of required replicas to populate the network 
for a given repulsing radius. 
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Figure 11: Source repartitions after replication 
with r = 25, maxScore — 250 and t = 16. 



Figure 11 shows how well the replicas are spread ac- 
cross the network. We measure the distance between 
the node hosting a replica and the closest node in the 
network hosting the same replica. This measurement 
has been collected just after a replication procedure has 
been performed. 

Replicas are better distributed on the geode topol- 
ogy. Nevertheless, the experimental topology exhibits 
a density mostly confined into a small bounded interval 
([20, 25]). This shows the effectiveness of the replication 
procedure applied on our dedicated topology. 

5.2 Adaptation to the number of nodes 

We simulate failure or entering of new nodes to arti- 
ficially increase or decrease the total number of nodes 




Figure 12: Evolution of the number of replicas 
according to the network's size. MaxScore = 10 x 
r. 



Figures 12 and 13 show the evolution of the number 
of replicas (bright line) according to the entering or de- 
parture of nodes in the network (dark line). Setting 
used are r = 30 and t — 16. Again, we suppose that 
there is always enough space in the cache of nodes and 
that caches are never purged. 
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Figure 13: Evolution of the number of replicas 
according to the network's size. MaxScore = 20 x 
r. 

Although there are small oscillations, we can see that 
the overall quantity of replicas remains proportional 
to the number of nodes in the network. As awaited, 
maxScore controls the variation of reactivity of the 
replication algorithm described in section 3.1.2. 

5.3 Replication of rare information 

In this experiment, we have generated a 10 000 nodes 
network, storing data on nodes according to a distribu- 
tion close to the one measure by [7]: few data is highly 
replicated while the majority of data has none or few 
replicas. On the 100 000 items spread across the net- 
work, half of them are not replicated and two third have 
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at most one replica. All items have the same size. Each 
cache is sized according to the number of item allocated 
for a node. Setting used are r = 30 and t = 4. 
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Figure 14: Evolution of the number of replica for 
each item in the network. Items are ranked by 
increasing rarity. Only rare data is replicated. 

The figure 14 confirms that only rare data is repli- 
cated. For highly replicated items (10 replicas or more), 
the amount of replicas remains the same. 

5.4 Answering speed 

Since super-peers hold indexed information of their 
sub-peers, we measure the answering speed by counting 
the number of super-peers visited by a random walker 
in the super-peer network before the query is answered. 
We generate a 100 000 nodes network with the exper- 
imental topology and spread one item and its replicas 
accross the network. 

The fact that it is quite unlikely that a super-peer 
hosts several copies of the same piece of data indexed 
by distinct sub-peers, slighly increases the answering 
speed (section 4.4.1). In this experiment, the number 
of sub-peers a super-peer can manage is 10 times less 
than its capability. Using distribution of figure 8, the 
simulated network contains on average 1.2% of super- 
peers. 




Hop count limit 

Figure 15: Query answer speed. 



The figure 15 shows the number of queries fullfillcd 
according to the maximum number of nodes that can be 
visited by the walker (hop count limit). For a 100 000 
nodes network containing on average 1.2% of super- 
peers, the search space is reduced to 1200 nodes. The 
use of super-peers greatly increases answering speed, 
even if low replicated data are still difficult to locate. 

6. CONCLUSION AND FUTURE WORK 

We introduced a data P2P architecture that uses stor- 
age space offered by users to increase data availability 
through wise replication. The replication scheme re- 
lies on exhaustive exploration of the neighborhood of 
each node, adapting dynamically the number of data 
replica. The agent-based regulation mechanisms pre- 
sented give the system some homeostasis properties: the 
global number of replicas remains stable and wisely dis- 
patched whithout any supervised nor global awarness 
or the network state when a node enters or leaves the 
network. The spiral walk algorithm is supported by a 
near-regular triangular closed mesh : this topology is 
maintained by a rule-based system that keeps repairing 
the mesh locally. 

The search cost introduced by the fact that the sys- 
tem makes no assumption on the type of query used, is 
lowered by a dedicated system layout based on the use 
of super-peers. 

We have presented an exploration scheme that gen- 
erates a number of messages lineary proportional to the 
number of visited nodes. However, when the topology 
is well suited for replication, it generates one walker, 
whereas it become parallelised only when the topology 
is not suited for replication. We are currently look- 
ing toward the possibility to launche many walkers in 
parallel that will explore their own area of the neigh- 
borhood of any anode. This parallelisation could speed 
up neighborhood exploration, reducing the possibility of 
node departure while performing the exploration, thus 
increasing system robustness. 

Another interesting feature that could be added is 
the use of erasure codes [21] for data replication in the 
cache offered by users. 
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